Interesting Paper Exploring Prompt Injection

This is a fascinating explotation of how LLMs fall for prompt injection attacks. It turns out that they learn to recognize the style of text in different role/instruction blocks, and not just the tags. Their conclusion: Role tags were a formatting trick that became the security architecture and the …

View Post on Scour ↗

More from Schneier on Security on Scour.

Covers 3 stories including A Theory of Why Prompt Injection Works